The following picture is of a feedforward network with

• A single input x
• Two hidden layers
• A singe output

The first hidden layer has M neurons.

The second hidden layer has N neurons.

Solution

To calculate the number of multiplications needed for a single feedforward pass, we can break down the network to three steps:

• Step 1: From the single input to the first hidden layer
• Step 2: From the first hidden layer to the second hidden layer
• Step 2: From the second hidden layer to the single output

Step 1

The single input is multiplied by a vector with M values. Each value in the vector will represent a weight connecting the input to the first hidden layer. Therefore, we will have M multiplication operations.

Step 2

Each value in the first hidden layer (M in total) will be multiplied by a vector with N values. Each value in the vector will represent a weight connecting the neurons in the first hidden layer to the neurons in the second hidden layer. Therefore, we will have here M times N calculations, or simply MN multiplication operations.

Step 3

Each value in the second hidden layer (N in total) will be multiplied once, by the weight element connecting it to the single output. Therefore, we will have N multiplication operations.

In total, we will add the number of operations we calculated in each step: M+MN+N .